Performance Evaluation of User Modeling Servers under Real-World Workload Conditions

Before user modeling servers can be deployed to real-world application environments with potentially millions of users, their runtime behavior must be experimentally verified under realistic workload conditions to ascertain their satisfactory performance in the target domain. This paper discusses performance experiments which systematically vary the number of profiles available in the user modeling server, and the frequency of page requests that simulated users submit to a hypothetical personalized website. The parameters of this simulation are based on empirical web usage research. For small to medium sized test scenarios, the processing time for a representative mix of user modeling operations was found to only degressively increase with the frequency of page requests. The distribution of the user modeling server across a network of computers additionally accelerated those operations that are amenable to parallel execution. A large-scale test with several million active user profiles and a page request rate that is representative of major websites confirmed that the user modeling performance of our server will not impose a significant overhead for a personalized website. It also corroborated our earlier finding that directories provide a superior foundation for user modeling servers than traditionally used data bases and knowledge bases.